Diffusion models have quickly become the go-to paradigm for generative modelling of perceptual signals (such as images and sound) through iterative refinement. Their success hinges on the fact that the underlying physical phenomena are continuous. For inherently discrete and categorical data such as language, various diffusion-inspired alternatives have been proposed. However, the continuous nature of diffusion models conveys many benefits, and in this work we endeavour to preserve it. We propose CDCD, a framework for modelling categorical data with diffusion models that are continuous both in time and input space. We demonstrate its efficacy on several language modelling tasks.

正如人类和动物在自然世界中学习的那样，它们会遇到远非统一的实体，情况和事件的分布。通常，经常遇到相对较小的经历，而许多重要的体验很少发生。现实的高度紧密，重尾的本质构成了人类和动物通过不断发展的专业记忆系统所面临的特殊学习挑战。相比之下，大多数流行的RL环境和基准涉及属性，对象，情况或任务的大致变化。 RL算法将如何在环境特征分布的世界（如我们的）中表现出较不统一的分布？为了探讨这个问题，我们开发了三个互补的RL环境，在这些环境中，代理商的经验根据Zipfian（离散幂定律）分布而变化。在这些基准上，我们发现标准的深入RL体系结构和算法获得了对常见情况和任务的有用知识，但无法充分了解稀有的情况。为了更好地了解这一失败，我们探讨了如何调整当前方法的不同方面，以帮助提高罕见事件的性能，并表明RL目标功能，代理商的记忆系统和自我监督的学习目标都可以影响代理商的能力从罕见的体验中学习。这些结果共同表明，从偏斜的经验中进行强大的学习是应用模拟或实验室以外的深度RL方法的关键挑战，而我们的Zipfian环境为衡量未来的进步朝着这一目标提供了基础。

We introduce Bootstrap Your Own Latent (BYOL), a new approach to self-supervised image representation learning. BYOL relies on two neural networks, referred to as online and target networks, that interact and learn from each other. From an augmented view of an image, we train the online network to predict the target network representation of the same image under a different augmented view. At the same time, we update the target network with a slow-moving average of the online network. While state-of-the art methods rely on negative pairs, BYOL achieves a new state of the art without them. BYOL reaches 74.3% top-1 classification accuracy on ImageNet using a linear evaluation with a ResNet-50 architecture and 79.6% with a larger ResNet. We show that BYOL performs on par or better than the current state of the art on both transfer and semi-supervised benchmarks. Our implementation and pretrained models are given on GitHub. 3 * Equal contribution; the order of first authors was randomly selected.

We study the learning dynamics of self-predictive learning for reinforcement learning, a family of algorithms that learn representations by minimizing the prediction error of their own future latent representations. Despite its recent empirical success, such algorithms have an apparent defect: trivial representations (such as constants) minimize the prediction error, yet it is obviously undesirable to converge to such solutions. Our central insight is that careful designs of the optimization dynamics are critical to learning meaningful representations. We identify that a faster paced optimization of the predictor and semi-gradient updates on the representation, are crucial to preventing the representation collapse. Then in an idealized setup, we show self-predictive learning dynamics carries out spectral decomposition on the state transition matrix, effectively capturing information of the transition dynamics. Building on the theoretical insights, we propose bidirectional self-predictive learning, a novel self-predictive algorithm that learns two representations simultaneously. We examine the robustness of our theoretical insights with a number of small-scale experiments and showcase the promise of the novel representation learning algorithm with large-scale experiments.

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.

自动生物医学图像分析的领域至关重要地取决于算法验证的可靠和有意义的性能指标。但是，当前的度量使用通常是不明智的，并且不能反映基本的域名。在这里，我们提出了一个全面的框架，该框架指导研究人员以问题意识的方式选择绩效指标。具体而言，我们专注于生物医学图像分析问题，这些问题可以解释为图像，对象或像素级别的分类任务。该框架首先编译域兴趣 - 目标结构 - ，数据集和算法与输出问题相关的属性的属性与问题指纹相关，同时还将其映射到适当的问题类别，即图像级分类，语义分段，实例，实例细分或对象检测。然后，它指导用户选择和应用一组适当的验证指标的过程，同时使他们意识到与个人选择相关的潜在陷阱。在本文中，我们描述了指标重新加载推荐框架的当前状态，目的是从图像分析社区获得建设性的反馈。当前版本是在由60多个图像分析专家的国际联盟中开发的，将在社区驱动的优化之后公开作为用户友好的工具包提供。

数据增强是自然语言处理（NLP）模型的鲁棒性评估的重要组成部分，以及增强他们培训的数据的多样性。在本文中，我们呈现NL-Cogmenter，这是一种新的参与式Python的自然语言增强框架，它支持创建两个转换（对数据的修改）和过滤器（根据特定功能的数据拆分）。我们描述了框架和初始的117个变换和23个过滤器，用于各种自然语言任务。我们通过使用其几个转换来分析流行自然语言模型的鲁棒性来证明NL-Upmenter的功效。基础架构，Datacards和稳健性分析结果在NL-Augmenter存储库上公开可用（\ url {https://github.com/gem-benchmark/nl-augmenter}）。

尽管自动图像分析的重要性不断增加，但最近的元研究揭示了有关算法验证的主要缺陷。性能指标对于使用的自动算法的有意义，客观和透明的性能评估和验证尤其是关键，但是在使用特定的指标进行给定的图像分析任务时，对实际陷阱的关注相对较少。这些通常与（1）无视固有的度量属性，例如在存在类不平衡或小目标结构的情况下的行为，（2）无视固有的数据集属性，例如测试的非独立性案例和（3）无视指标应反映的实际生物医学领域的兴趣。该动态文档的目的是说明图像分析领域通常应用的性能指标的重要局限性。在这种情况下，它重点介绍了可以用作图像级分类，语义分割，实例分割或对象检测任务的生物医学图像分析问题。当前版本是基于由全球60多家机构的国际图像分析专家进行的关于指标的Delphi流程。

Array programming provides a powerful, compact, expressive syntax for accessing, manipulating, and operating on data in vectors, matrices, and higher-dimensional arrays [1]. NumPy is the primary array programming library for the Python language [2,3,4,5]. It plays an essential role in research analysis pipelines in fields as diverse as physics, chemistry, astronomy, geoscience, biology, psychology, material science, engineering, finance, and economics. For example, in astronomy, NumPy was an important part of the software stack used in the discovery of gravitational waves [6] and the first imaging of a black hole [7].Here we show how a few fundamental array concepts lead to a simple and powerful programming paradigm for organizing, exploring, and analyzing scientific data. NumPy is the foundation upon which the entire scientific Python universe is constructed. It is so pervasive that several projects, targeting audiences with specialized needs, have developed their own NumPy-like interfaces and array objects. Because of its central position in the ecosystem, NumPy increasingly plays the role of an interoperability layer between these new array computation libraries.

Making histopathology image classifiers robust to a wide range of real-world variability is a challenging task. Here, we describe a candidate deep learning solution for the Mitosis Domain Generalization Challenge 2022 (MIDOG) to address the problem of generalization for mitosis detection in images of hematoxylin-eosin-stained histology slides under high variability (scanner, tissue type and species variability). Our approach consists in training a rotation-invariant deep learning model using aggressive data augmentation with a training set enriched with hard negative examples and automatically selected negative examples from the unlabeled part of the challenge dataset. To optimize the performance of our models, we investigated a hard negative mining regime search procedure that lead us to train our best model using a subset of image patches representing 19.6% of our training partition of the challenge dataset. Our candidate model ensemble achieved a F1-score of .697 on the final test set after automated evaluation on the challenge platform, achieving the third best overall score in the MIDOG 2022 Challenge.